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Abstract 

Background: An exponential increase in the number of systematic reviews published, and constrained resources 
for new reviews, means that there is an urgent need for guidance on explicitly and transparently integrating 
existing reviews into new systematic reviews. The objectives of this paper are: 1) to identify areas where existing 
guidance may be adopted or adapted, and 2) to suggest areas for future guidance development. 

Methods: We searched documents and websites from healthcare focused systematic review organizations to 
identify and, where available, to summarize relevant guidance on the use of existing systematic reviews. We 
conducted informational interviews with members of Evidence-based Practice Centers (EPCs) to gather experiences 
in integrating existing systematic reviews, including common issues and challenges, as well as potential solutions. 

Results: There was consensus among systematic review organizations and the EPCs about some aspects of 
incorporating existing systematic reviews into new reviews. Current guidance may be used in assessing the 
relevance of prior reviews and in scanning references of prior reviews to identify studies for a new review. However, 
areas of challenge remain. Areas in need of guidance include how to synthesize, grade the strength of, and present 
bodies of evidence composed of primary studies and existing systematic reviews. For instance, empiric evidence is 
needed regarding how to quality check data abstraction and when and how to use study-level risk of bias 
assessments from prior reviews. 

Conclusions: There remain areas of uncertainty for how to integrate existing systematic reviews into new reviews. 
Methods research and consensus processes among systematic review organizations are needed to develop 
guidance to address these challenges. 
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Background 

Evidence -informed decisions in health care rely on the 
explicit consideration of available evidence through the 
conduct of systematic reviews. In the context of exponen- 
tial increase in the number of systematic reviews pub- 
lished, constrained resources for new reviews, and the 
increase in the need to update reviews, there is an urgent 
need for detailed methods to explicitly and transparently 
integrate existing reviews into systematic reviews. 

Since 1997, the US Agency for Healthcare Research and 
Quality (AHRQ) has supported the development of system- 
atic reviews about health care through its Evidence-based 
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Practice Center (EPC) program. These reviews generally 
address multiple questions and are conducted following 
standard methods. A number of years ago, researchers from 
several EPCs recognized that reviewers were increasingly 
faced with decisions about whether and how to incorporate 
existing systematic reviews into new systematic reviews. 
This group collaborated to develop guidance for the use 
of existing systematic reviews in new reviews [1]. As part 
of the work in developing the EPC Methods Guide, a 
workgroup was subsequently formed by the AHRQ on the 
same topic, and guidance that generally reflected the prior 
work was issued [2], 

The guidance we provided in the EPC Methods Guide 
noted the need for additional work on specific methods 
for using existing systematic reviews in reviews. Our 
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intervening experience in implementing this guidance 
has also identified additional challenges, particularly in 
grading the strength of evidence and presenting the 
body of evidence when existing systematic reviews form 
part of the overall evidence base in a new systematic 
review. 

The lack of guidance has led to variable perspectives 
about and usage of existing systematic reviews across 
the EPC program. In particular, uncertainty and lack of 
guidance has led some EPC researchers to avoid inte- 
grating existing systematic reviews within the strength of 
evidence judgments and summaries in new reviews. Omit- 
ting existing reviews from consideration when assessing 
the strength of the evidence is unsatisfying for the EPCs 
and for the audiences of our reports, as recently noted in 
recent feedback by the Eisenberg Center [3]. Some EPCs 
have chosen to integrate existing systematic reviews into 
the strength of evidence ratings, but without guidance on 
how to implement such integration, the process has been 
inconsistent. 

The objectives of our workgroup of EPC members, 
and of this white paper, are: 1) to identify areas where 
existing guidance may be adopted or adapted, and 2) to 
suggest approaches and focused areas for future work and 
additional guidance needs. 

Methods 

Key methodological steps in deciding whether and how 
to use existing systematic reviews in new systematic re- 
views of health care questions are illustrated in Figure 1 
and in the accompanying Table 1 (adapted from Whitlock 
et al [1]). These steps include five, non-mutually exclusive 
options for how existing systematic reviews might be used. 
In this paper, these methodological areas were used to 
classify existing guidance from other health-focused sys- 
tematic review organizations and to focus the discussion 
with directors and staff at EPCs. Table 1 provides defini- 
tions for the key steps in Figure 1. 

Guidance summary 

Documents and websites from healthcare focused sys- 
tematic review organizations were manually searched for 
available guidance on the use of existing systematic re- 
views in new systematic reviews. Where available, rele- 
vant guidance was extracted and summarized for each of 
the methodological steps illustrated in Figure 1. 

Evidence-based practice center discussions 

Individual workgroup members held discussions with 
volunteer EPC members including EPC directors, associate 
directors, and project managers. An interview guide, in- 
cluding 11 open-ended questions, facilitated the informal 
discussions. The goals of the discussions were to gather in- 
sights from the experiences of EPCs in integrating existing 



systematic reviews into new reviews and to identify com- 
mon issues and challenges, as well as potential solutions. 
Individual workgroup members conducted discussions with 
15 EPC members representing 10 of 11 EPCs. A list of par- 
ticipating EPC members can be found in Appendix A. Each 
call was, with permission of those on the call, recorded and 
then transcribed. Major themes from the interviews were 
identified by workgroup members through review of the 
transcripts. These themes were then compared to the exist- 
ing guidance to identify areas of overlap and need. 

Assessment of areas of need 

Workgroup members assessed the sufficiency of cur- 
rently available guidance based on the needs expressed 
by EPCs. Through individual and group deliberation, the 
workgroup determined which methodological steps on 
the use of existing systematic reviews need additional 
guidance. For each area where additional guidance is 
needed, the workgroup generated suggestions for how 
to develop this guidance. 

Results 

Guidance summary 

Overall, about half of the systematic review organizations 
provided some direction on the use of existing systematic 
reviews in new reviews. Table 2 shows the list of organiza- 
tions for which websites were manually searched. A check 
mark under the area of guidance denotes a comment on 
the use of existing systematic reviews, though not ne- 
cessarily detailed guidance. Listed organizations with no 
check marks are organizations for which guidance on 
the use of existing systematic reviews was not found. 

Evidence-based practice center discussions 

Overall themes were identified from the discussions with 
EPCs: 

• EPCs most commonly used existing reviews as a 
source of relevant literature and as context for the 
introduction or discussion section of reviews. 

• Existing reviews were most useful when key 
questions and/or PICOTS-SD (population, intervention, 
comparator, outcome, time frame, setting, and study 
design) matched or when they addressed a specific 
subquestion of the new review. 

• Using existing reviews was often more resource 
intensive than completing a review from scratch. 

• EPCs expressed that they often did not trust aspects 
of reviews conducted by others. 

• When relevant and rigorous, incorporating prior 
reviews into the review being undertaken by the 
EPC was clearly valuable in at least two instances: 1) 
allowing larger scope of the review being undertaken 
without additional resources, or 2) providing 
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Yes 



Step 2. Assess relevance 
o Questions 
o Methods 
o Search dates 




No 



Yes 



Step 3. Assess quality of 
existing SR(s) 




No 



Step 4. Determine appropriate 
use and incorporate existing 
SR(s) 



AND 



Step 5. Report methods and results 
from using existing SR(s) 



Stop. Proceed with SR of 

primary evidence 

Use "almost" relevant SRs to 

frame and provide context 

(Contextual Use) 

Scan References of "almost" 

relevant SRs to check new 

search results 



Scan references, check new search 
results 



Use existing search 



Use existing data abstraction, study-level 
risk of bias assessments and/or synthesis 



Use complete review 



Figure 1 Methodological steps in using existing systematic reviews (SRs). 



summarized evidence when a new in depth review 
of primary literature would not be feasible (for 
example, existing reviews provide individual patient 
data analysis or include hundreds of trials, 
supplemented by author-provided data). 

EPCs have used existing reviews in various ways, most 
commonly as a source of relevant literature, allowing 
them to reduce the extent of searching to locate primary 
literature or to check completeness of primary literature 
search strategies. Additionally, prior reviews are often 



used to provide context for the introduction or discussion 
sections of a review. At a minimum, most EPC members 
feel that it is necessary to acknowledge other systematic 
reviews and to put the findings of the current review into 
the context of other systematic reviews, particularly in the 
case of disagreements or controversy. 

EPC members noted that methods determining when 
and how to use an existing review are highly dependent 
on the topic and scope of the new report. There are cer- 
tain instances when it may be most feasible to use an 
existing review as evidence in a new review. For example, 
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Table 1 Definitions of terms used in Figure 1 

Locate existing SR(s) 

A defined and reproducible approach to efficiently identify existing 
systematic reviews for possible use in conducting a newly proposed 
systematic review, including updates. 

Assess relevance 

Methods by which existing systematic reviews identified in Step 1 can 
be evaluated as to whether they are similar enough to the newly 
proposed review to obviate the need for conducting one or several 
steps in undertaking the newly proposed review. Relevance evaluation 
considers how well the existing reviews' research questions and 
inclusion/exclusion criteria for population, interventions, comparators, 
outcomes, settings, and study designs match those of the new 
systematic review, and how recently existing reviews' literature searches 
were concluded. Use 'almost' relevant SR(s) when selecting, developing, 
and/or refining questions and providing context for a newly proposed 
systematic review (that is, Contextual Use) and scan references to check 
new search results. 

Assess quality of existing SR(s) 

Methods by which relevant existing systematic reviews can be 
evaluated for quality of methodological approach, using AMSTAR or 
other commonly used tools, with a focus on potentially incorporating 
only reviews meeting certain quality criteria into the proposed review 
[4,5]. 

Determine appropriate use and incorporate existing SR(s) 

Methods by which to determine appropriate uses for relevant, high-quality 
existing systematic reviews in the proposed review. Incorporate existing 
SR(s) or use information from existing systematic reviews to supplement or 
supplant one or more activities that would be conducted from scratch for 
the newly proposed review. Use of existing systematic reviews may 
include: 1) using the existing systematic review(s)' listing of included studies 
as a quality check for the literature search and screening strategy conducted 
for the new review (Scan References); 2) using the existing systematic 
review(s) to completely or partially provide the body of included studies for 
one or more key questions in the new review (Use Existing Search); 3) using 
the data abstraction, risk of bias assessments, and/or analyses from existing 
systematic reviews for one or more key questions in the new review (Use 
Data Abstraction/Syntheses); or 4) using the existing systematic review(s), 
including conclusions, to fully or partially answer one or more key 
questions in the new review (Use Complete Review). 

Report methods and results from using existing SR(s) 

Standards for reporting the rationale for incorporating existing 
systematic reviews and the methods by which specific existing 
systematic reviews were located, assessed, and incorporated into the 
current systematic review. Standards for reporting results in the current 
systematic review that rely on evidence and/or analysis from existing 
systematic reviews and caveats or limitations associated with that 
approach; guidance about discussing how findings of the current 
systematic review compare and contrast with those of existing relevant 
systematic reviews. 

AMSTAR, A Measurement Tool to Assess Systematic Reviews; SR, 
systematic review. Adapted from Whitlock et al. [1]. 

existing reviews are more likely to be used as evidence for 
a new review when the key questions match and when the 
review questions are concerned primarily with one or a 
few clearly and specifically defined and measured out- 
comes; existing systematic reviews are fruitful to incorpor- 
ate to answer contextual questions that do not demand a 
full systematic review, but are important for decision 
makers when considering the evidence base. Conversely, 
EPC members indicated there are other instances when it 



would not make sense to use an existing review, such as 
when there are few studies on a given topic or when there 
are multiple conflicting systematic reviews. When incorp- 
orating existing reviews into a new or updated review, 
EPC members most often described qualitative or narrative 
incorporation of the existing reviews, noting that quantita- 
tive combination of findings (without going back to all pri- 
mary studies) is more difficult and potentially introduces 
error, and thus, is less commonly done. 

EPC members varied in their thoughts and experiences 
about the efficiency of incorporating existing reviews into 
new reviews. Most thought that, although this process 
theoretically should result in efficiency gains, challenges 
that arise when trying to use reviews often negate any in- 
creased efficiency. These challenges arise from the need to 
understand and qualify the methods for existing system- 
atic reviews as intensively as primary studies. Thus, EPC 
members often described experiences where using a prior 
systematic review resulted in as or more intensive resource 
requirements as completing a new review of primary litera- 
ture. A particular concern were instances in which stake- 
holders who nominated the review requested a new review 
after seeing the results of incorporation of existing reviews. 
It is hard to estimate how much work will be required to 
clarify the relevance or quality of existing systematic re- 
views, since effort depends on the volume of existing sys- 
tematic reviews, as well as issues specific to the match 
between existing reviews and the review being undertaken; 
these include whether the key questions are an exact 
match or how the existing systematic review authors 
approached important methods, such as strength of evi- 
dence grading or risk of bias assessment. In a number 
of cases, EPC members described eventually having to 
conduct much of the process from scratch in spite of 
completing the additional step of in-depth examination 
of existing reviews. EPC members voiced concern about 
using this process based on the potential (although 
often unrealized) benefit of efficiency, while running an 
unclear risk of error or reduced quality from relying on 
unverifiable work from others. Additionally, EPC mem- 
bers described discomfort with the lack of guidance in 
this area, noting inconsistency across EPCs in how this 
process is approached and hesitancy to engage in such 
a process without explicit direction in the EPC Methods 
Guide. 

Although EPC members felt that it was often difficult 
to gain efficiency incorporating existing reviews in new 
reviews, they acknowledged other potential benefits to 
this process. Some described that including existing re- 
views sometimes enables them to cover a wider range of 
questions and elements of questions (as denoted by PI- 
COTS) when existing systematic reviews address import- 
ant aspects of new review key questions. EPC members 
suggested that this breadth is often desirable, but may 



Table 2 Available guidance/comment on the use of existing systematic reviews 



Guidance areas 

Locating and Assessing Assessing Determining Determining Determining Determining Determining Methods/ 

defining relevance review quality use: use: use: use: use: Results 

appropriate use Scanning Search Risk of bias Data Synthesis reporting 

references strategy assessment abstraction 

AHRQ Evidence-based Practice Center Program -/ -/-/ S </ S S </ </ 

(EPC program) 

Methods Guide for Effectiveness and Comparative 
Effectiveness Reviews [2] 

Reporting the Findings of Updated Systematic J 
Reviews of Comparative Effectiveness [6] 

Canadian Agency for Drugs and Technology in 
Health (CADTH) 

Guidelines for Authors of CADTH Health 
Technology Assessment Reports [7] 

Cochrane Collaboration / -/</ S </ S S </ </ 

Cochrane Handbook for Systematic Reviews of 
Interventions [8] 

Danish Centre for Health Technology </ -/ / 

Assessment (DACEHTA) 

Health Technology Assessment Handbook [9] 

European Collaboration for Health Technology </ / / 

Assessment (ECHTA) 

Best Practice in Undertaking and Reporting Health 
Technology Assessments [1 0] 

Health Technology Assessment International 

Institute of Medicine 

Finding What Works in Health Care: Standards for 
Systematic Reviews [11] 

Institute for Quality and Efficiency in Health </ / / 

Care (IQWiG) 

General Methods [1 2] 

National Institute for Health and Care / -/ / -/ 

Excellence (NICE) 

The Guidelines Manual Draft for Consultation [1 3] 

York Centre for Reviews and Dissemination: </ / -/ 

Systematic Reviews (CRD) 

CRD's Guidance for Undertaking Reviews in 
Healthcare [14] 
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not be possible if independent review of the primary lit- 
erature is required due to time and resource limitations. 
Similarly, important evidence to answer some key ques- 
tions may include individual patient data meta-analyses 
or data from a large number of trials, representing work 
from systematic reviewers that would be very difficult or 
impossible to reproduce. 

Overall, EPC members noted that more guidance is 
needed for using existing reviews in new systematic re- 
views. Most felt that detailed, specific, step-by-step guid- 
ance may not be feasible, but that some further articulation 
of important areas to consider, with clear worked examples, 
would be helpful. Commonly highlighted areas in need of 
additional guidance include: 

1. Providing principles or criteria for when a new 
review adds value to a field with many existing 
reviews. 

2. Providing templates or advisory considerations for 
construction of evidence tables for reviews 
combining primary and secondary (systematic 
review-level) evidence. 

3. Reporting guidelines for clearly communicating the 
methods for locating, selecting, and deciding how 
best to utilize existing systematic reviews. 

4. Methods that limit the potential for bias in selecting 
reviews to incorporate from among multiple existing 
reviews. 

5. Guidance on methods that limit the potential for 
bias in incorporating selected portions of a 
review. 

6. Qualitative and quantitative methods for 
summarizing bodies of evidence that include a 
systematic review as the only or as one source of 
evidence. 

7. More robust means for quality rating of existing 
systematic reviews (beyond AMSTAR [4]). 

8. Specific methods to grade strength of evidence 
for bodies of evidence that include a systematic 
review as the only or as one source of evidence. 

Guidance addressing the first item listed is more of a 
scoping question prior to the initiation of a new review 
(for example, within the Topic Refinement stage of the 
EPC programs current processes). In this paper, we as- 
sume that a new review has been started and is the review 
authors are considering incorporating existing reviews. 
The remaining challenges in using existing reviews are 
discussed below and fall within each of the methodological 
areas presented in Figure 1. A summary of the existing 
guidance for each area is presented along with an as- 
sessment of future guidance needs. 



Discussion 

Methodological areas: assessment of areas of need 
Locating existing systematic reviews 

Available guidance Several organizations present guid- 
ance on locating existing systematic reviews, including the 
EPC program and the Cochrane Collaboration (see Table 3). 
These groups recommend using specific databases and 
search filters to aid in locating existing systematic reviews. 
Commonly recommended databases include: Database of 
Abstracts of Reviews of Effects (DARE), Cochrane Data- 
base of Systematic Reviews (CDSR), Health Technology 
Assessment Database, MEDLINE and Embase. Some or- 
ganizations promote limiting searches for existing system- 
atic reviews to selected sources (for example, CDSR), with 
the idea that these systematic reviews would be expected 
to meet sufficient quality standards. We identified no cri- 
teria for selecting systematic reviews. 

Evidence-based practice center discussions EPC mem- 
bers expressed concerns about locating reviews based on 
very limited sources, for example, only searching for EPC 
or Cochrane reviews. Some EPC members were uncom- 
fortable with any type of selective search instead of doing 
a broader, if not comprehensive, search for existing sys- 
tematic reviews. However, other EPC members felt it 
could be appropriate to selectively use one or two earlier 
reviews without having to review all of the available prior 
reviews, pointing out that the scientific rationale and pur- 
pose in conducting a systematic search for existing system- 
atic reviews is different than when searching for primary 
studies. Currently, there is a lack of consensus and limited 
guidance concerning how to adequately locate and trans- 
parently select and use only a subset of reviews. 

Assessment Current EPC guidance on locating existing 
systematic reviews states that EPCs should conduct a 
targeted search of a higher yield database, which in- 
cludes output from the EPC program, MEDLINEs Top 
120 Index Medicus Journals, Health Technology Assess- 
ments, CDSR and DARE. EPC program guidance sug- 
gests that identifying existing systematic reviews may be 
done separately or be completed as part of the broad 
search for primary literature to answer key questions. 
EPC members shared concern about the extensive effort 
that may be required to search for and locate all reviews 
and assess their quality. Instead, given the purpose of lo- 
cating existing systematic reviews, the ideal search might 
locate only highly relevant, well-done, very recent, exist- 
ing systematic reviews that are more likely to allow the 
current reviewer to leverage the prior work. Some mem- 
bers proposed that future guidance further limit the search 
for earlier reviews to sources that may have greater likeli- 
hood of identifying higher quality and better reported re- 
views, such as the EPC and Cochrane databases. This 



Table 3 Guidance summary 



AHRQ Evidence-based Practice Center Program (EPC program) 



Cochrane collaboration 



Danish Centre for Health Technology 
Assessment (DACEHTA) 



Locating 



Assessing 
Relevance 



Assessing 

Review 

Quality 



Determining 
use: 

Scanning 
References 



Two strategies are recommended for identifying existing systematic reviews for a CER. The 
first strategy is to perform a targeted search of a higher yield database, which includes 
output from the Evidence-based Practice Center program, MEDLINE'sTop 120 Index Medicus 
Journals, Health Technology Assessments, Cochrane Database of Systematic Reviews and Data- 
base of Abstracts and Reviews of Effects. The second strategy is to identify systematic reviews 
during a broad de novo literature search. 



An existing systematic review should be used with the intent to answer parts or all of 
specific key questions. PICOTS-SD must be considered for relevance of existing systematic 
reviews. Reviews that are partially relevant may be useful for background or checking 
references. An initial screening for relevance should be performed, considering the 
timeliness of the review's literature search. It is recommended to bridge any search date 
that ended more than one year from the time the systematic review is identified. If a 
review is outdated but still desired to be used, an update of the search should be done. 

In the second stage of screening, the review's PICOTS-SD elements should be compared to 
those in the new review protocol for relevance. If these elements are poorly reported, the 
review should not consider including the existing review. 



Systematic reviews can be located through 
CDSR, DARE and HTA database. MEDLINE and 
EMBASE can also be used to search for 
systematic reviews. In MEDLINE, most review 
articles can be found under the publication 
Term 'Meta-analysis' and in EMBASE, the 
thesaurus term 'Systematic Review' can be 
used. Specific search strategies can be used to 
identify systematic reviews in MEDLINE and 
EMBASE. Additionally, systematic reviews can 
be identified through search services such as 
Turning Research into Practice (TRIP). 

In an Overview, primarily only Cochrane 
Intervention reviews should be included, but 
other reviews may be included occasionally 

In an Overview, included reviews should be 
assessed using specific criteria. 
Considerations include whether a review is 
up-to-date and if there are specific limita- 
tions for the objectives of the Overview. 



Only reviews of high quality should be included in a CER. Two independent reviewers should 
assess for quality and methods for resolving discrepancies should be reported. A quality rating 
instrument should be used to addresses all aspects of the review that will be incorporated 
into the CER. Both the methods used to minimize bias and the reporting should be assessed. 
QUOROM (PRISMA) is a checklist that can be used to assess the reporting of systematic 
reviews. As a common starting point, the AMSTAR tool should be used to assess the quality of 
reviews. Reproducibility and application of inclusion and exclusion criteria should be 
confirmed. As some limitation to AMSTAR exists, it is recommended to describe implications 
of potential methodological flaws instead of relying on numerical scores. 

The list of included articles from an existing review can be used in a CER if methods for 
identifying articles are of adequate quality. 



Secondary studies (for example, systematic 
reviews, HTA reports, and clinical guidelines) 
should be located to determine if key 
questions have already been answered. 
Secondary studies can be identified through 
several databases (for example, The HTA 
Database, Cochrane Database of Systematic 
Reviews, Database of Abstracts of Reviews of 
Effects, Guidelines International Network, 
National Guidelines Clearinghouse, Health 
Evidence Network, National Electronic 
Library for Health: Guidelines Finder, and 
Turning Research Into Practice). 



All evidence should be assessed for relevance to 
the topic. Identified articles should be compared 
to the focused question to determine if the 
article may answer the focus question. The 
literature can be divided into two groups 
(secondary studies and primary studies). If a large 
amount of evidence has been identified, it can 
be subdivided into groups based on presumed 
quality. The hierarchy of evidence is: 1) meta- 
analyses and systematic reviews (among others 
Cochrane reviews); 2) randomized controlled 
trials (RCTs); 3) non-randomized controlled trials; 
4) cohort studies; 5) case-control studies; 6) 
descriptive studies, limited series; and 7) position 
papers, non-systematic reviews, leading articles, 
expert opinions. 



Generally, selection criteria for a Cochrane 
Overview limits included reviews to 
Cochrane reviews. Non-Cochrane systematic 
reviews may be included if there are good 
quality reviews for which a Cochrane review 
is not available. 



Existing systematic reviews can be used as 
sources of relevant studies. References lists of 
systematic reviews can be searched to identify 
relevant articles. This should be done as an 
adjunct to other search methods as bias may 
be present in what studies were included in 
existing reviews. 



No guidance 



No guidance 



Table 3 Guidance summary (Continued) 



Determining 
use: Use 
Search 

Determining 
use: Risk of 
Bias 

Assessment 



Determining 
use: Data 
Abstraction 



Use 

Synthesis 



Report 

Methods/ 

Results 



Part or all of the search strategy may be used from an existing review if it is consistent Existing reviews may be a useful source No guidance 

with EPC program methods for finding evidence. A search strategy from an existing review information about search strategies 
can be used, followed by de novo analysis and synthesis of data. 

In order to use the risk of bias assessment from a systematic review, the methods used In an Overview, an assessment of the quality A quality assessment tool should be used to 



must be consistent with the EPC program methods guide. These methods include 
selection of design specific criteria for risk of bias assessment and use of appropriate tools. 



The data extraction tables may be used from an existing review, if they are deemed to be 
of adequate quality. However, if results of individual trials are not reported, the use of 
summary findings from an existing review may compromise transparency for a CER and 
this is not recommended. 



If an existing review is very similar to a CER in research questions and is high quality, the 
entirety or portions of the existing review may be incorporated. Summarized evidence for 
specific populations or interventions may be included in a CER. If summarized evidence is 
to be included, the existing review must have methods consistent with EPC program 
methods for finding evidence, assessing quality, grading the strength of evidence and 
other principles including conflicts of interest. Summarized evidence can also be 
incorporated with a de novo sensitivity analysis. 

If multiple high quality reviews are found, a single review can be chosen, which is most 
relevant and least biased, or a meta-review can be performed. 



If more than one high quality review is found with discordant findings, it may be an 
indication to start a de novo review on that key question. 

It is recommended to provide a summary table to show where existing review(s) were 
used to replace de novo processes. Summary tables of existing systematic reviews should 
be included to compare the reviews and should address any overlap (or lack thereof) in 
the primary research included in reviews. 

The discussion section should include a justification for using an existing systematic review 
and address any limitations. It is also important to compare findings from the CER with the 
findings from existing reviews. 

In an update, it is important to show explicitly what has changed from the previous report. 
The desired depth of information varies between users of reports. Review updates may be 
effectively presented in an executive summary with tables and figures, identifying and 
modifications followed by a full report for users who require further depth of information. 



of evidence should be done. If no 
assessment was done in an included 
systematic review, authors should perform 
the assessment. If a quality assessment was 
done in an included systematic review, 
authors should assess the judgments and 
ensure consistency between included 
reviews. 

In an update, data from new studies should No guidance 
be abstracted and included, if applicable. 

In an Overview, if necessary, authors may 
seek additional data or information from the 
authors of primary studies from included 
systematic reviews. 

In an update, data collected from new No guidance 

studies should be included and a new 
meta-analysis should be done. 



In an Overview, authors should reply on 
previous analyses when possible. If there are 
differences between reviews (for example, 
different populations or subgroups are 
analyzed), data may need to be reanalyzed. 



In an update, revision to text of the existing No guidance 

review will depend on the influence of the 

new data and results. If there is no change in 

the results, little revision to the text is 

required. However, some updates may require 

a change to the conclusion of a review which 

will require much modification of the text. It 

should be noted in the Abstract and 

Background that this is an update. A 'What's 

new' table should be completed and changes 

should be made to ensure no dates or other 

information is out of date. 



uniformly assess the quality of identified 
articles. Check list tools developed by 
different national centers (for example, SIGN, 
NICE, GRADE and Centre for Evidence-based 
Medicine, Oxford) can be used to assess 
quality. 



Table 3 Guidance summary (Continued) 



European Collaboration Health 
Technology Assessment (ECHTA) 



Institute for Quality and Efficiency in 
Health Care (IQWiG) 



National Institute for Health and Clinical 
Excellence (NICE) 



York Centre for Reviews and 
Dissemination: Systematic Reviews (CRD) 



Locating 



Assessing 
Relevance 



Assessing 

Review 

Quality 



In order to determine if key questions have 
already been answered, a search for 
previous HTA reports should be conducted. 
The search for previous reports should be 
systematic and well documented. 



All identified evidence should be assessed 
with pre-defined inclusion and exclusion cri- 
teria. Selection criteria should be developed 
from background information, research 
questions and the availability of evidence 
and should be defined prospectively to 
avoid bias in selection of evidence. Inclusion 
and exclusion criteria should cover patient 
characteristics, condition characteristics, 
technology aspects, methodological issues, 
outcomes measured, publication type. 

No guidance 



Different databases should be considered in 
locating systematic reviews that are used for 
primary literature. Databases that exclusively 
or mostly hold systematic reviews should be 
searched as well as biomedical databases 
(for example, MEDLINE and EMBASE), which 
also hold systematic reviews. 

Evidence scanning should be done 
continuously to identify systematic reviews 
that concern published or developing 
information products. Two people should 
regularly screen (CDSR, DARE, INAHTA, MORE 
and PubMed). Identified reviews that concern 
a product of the Institute can influence the 
updating process, including triggering an 
update or modifying the updating plan. 

No guidance 



Core and subject databases, including 
MEDLINE, EMBASE, Cochrane Database of 
Systematic Reviews, Database of Abstracts of 
Reviews of Effects, Cochrane Central Register 
of Controlled Trials, and Health Technology 
Assessment (HTA) database, should be 
searched for every review. For questions on 
effectiveness, a search should be done for 
systematic reviews, followed by randomized 
controlled trials, then cohort or case-control 
studies. Search filters are available to assist 
in identifying studies, including a search 
filter for systematic reviews. 



No guidance 



For systematic reviews to be used in a benefit 
assessment, they must be assessed for sufficient 
quality. They must 'show only a minimum risk 
of bias; present the evidence base in a 
complete, transparent and reproducible 
manner; and thus allow clear conclusions to be 
drawn'. The searches conducted in the 
systematic reviews must not contradict the 
methodology of the Institute. Quality 
assessment should be done with Oxman and 
Guyatt's quality index for systematic reviews or 
AMSTAR. Sponsors and authors' conflicts of 
interests should be documented and discussed 
for systematic reviews. 



To check if key questions have been answered 
by existing or ongoing reviews, a search for 
systematic reviews should be conducted. The 
Database of Abstracts of Reviews of Effects 
(DARE), and the Cochrane Database of 
Systematic Reviews (CDSR) should be 
searched. Additionally, NICE and NIHR HTA 
program Websites can be searched along with 
the Campbell Library of Systematic Reviews 
and the Evidence for Policy and Practice 
Information Centre's Database of Systematic 
and Non Systematic Reviews of Public Health 
Information (DoPHER). Guideline groups 
including NGC and SIGN can be searched for 
guidelines based on systematic review 
evidence. MEDLINE and other databases can 
be searched for previous reviews. 

No guidance 



Guidelines may contain reviews of evidence 
that are applicable to questions formulated by 
the guideline development group. These may 
be used as evidence if: 'they are assessed 
using the appropriate methodology checklist 
from this manual and are judged to be of 
high quality, they are accompanied by an 
evidence statement and evidence table(s), the 
evidence is updated according to the 
methodology for the exceptional update of 
NICE clinical guidelines'. 



Identified reviews should be assessed for 
quality. Quality reviews should have a well- 
defined question, comprehensive search, clear 
and appropriate selection or studies, unbiased 
processes for assessing study quality and 
extracting and synthesizing data. Checklists 
can be used to help in assessing quality of 
systematic reviews (for example, Oxman and 
Guyatt, Guidelines for reading literature 
reviews). 



Table 3 Guidance summary (Continued) 



Determining No guidance 
use: 

Scanning 
References 

Determining No guidance 
use: Search 

Determining All evidence should be critically assessed for 
use: Risk of quality. Checklists can be used for appraisal 
Bias of medical literature. All sources of 

Assessment information should be appraised for validity. 

No guidelines exist for assessing quality of 
sources of information other than medical 
literature, and this is a gap that future 
guidance needs to address. 

Determining No guidance 

use: Data 

Abstraction 



Determining No guidance 
use: 

Synthesis 



Report 

Methods/ 

Results 



No guidance 



If more than one systematic review of 
adequate quality is found to address a 
particular subject, additional quality assessment 
should be done. Items to compare include: 
content of the review, search strategy and 
date, sensitivity analysis, how bias is assessed 
and dealt with, and updating provisions. 

No guidance 



No guidance 



No guidance 



No guidance 



No guidance 



No guidance 



No guidance 



No guidance 



Results of systematic reviews should be 
summarized in tables if possible. If 
discordant results on the same outcome are 
found, possible explanations should be 
given. If it appears that a new benefit 
assessment based on primary studies would 
produce different results, a new assessment 
should be done. 



Other sources of literature include 
references lists of systematic reviews. 
References lists of existing reviews can be 
scanned to identify additional studies. 

No guidance 



No guidance 



If using reviews of evidence published in No guidance 

other guidelines, the guideline development 

group should create new evidence 

summaries or statements. The original 

evidence tables should be referenced with a 

direct link to the source if possible or a 

reference to the published document. 

Verbatim quotes of recommendations from 

other guidelines should not be used, unless 

the recommendations come from NHS 

policy or legislation. 

No guidance No guidance 



Original evidence tables from published No guidance 

guidelines should be referenced with a 

direct link to the source if possible or a 

reference to the published document. 

Verbatim quotes of recommendations from 

other guidelines should not be used, unless 

the recommendations come from NHS 

policy or legislation. 



CDSR, Cochrane Database of Systematic Reviews; CER, comparative effectiveness review; DARE, Database of Abstracts of Reviews of Effects; DoPHER, Database of Systematic and Non Systematic Reviews of Public 
Health Information; GRADE, Grading of Recommendations Assessment, Development and Evaluation; INAHTA, International Network of Agencies for Health Technology Assessment; MORE, McMaster Online Rating of 
Evidence; NIHR, National Institute for Health Research; NGC, National Guide Clearinghouse; NHS, National Health Service; NICE, National Institute for Health and Care Excellence; PICOTS-SD, population, intervention, 
comparator, outcome, time frame, setting, and study design; PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-Analyses; QUOROM, Quality of Reporting of Meta-analyses; SIGN, Scottish 
Intercollegiate Guidelines Network; TRIP, Turning Research into Practice. 
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would also, theoretically, negate the need to determine if a 
report is a systematic review. 

Assessing relevance 

Available guidance Several organizations present guid- 
ance on assessing the relevance of existing systematic re- 
views. European Collaboration for Health Technology 
Assessment (ECHTA) and Danish Centre for Health Tech- 
nology Assessment (DACEHTA) provide general guidance 
for assessing the relevance of all literature, but no guidance 
specific to existing reviews. Cochrane provides guidance for 
assessing the relevance of existing systematic reviews for 
use in an overview of reviews and specifically recommends 
that reviewers consider if a review is up-to-date (up-to-date 
is not defined in this context, but for Cochrane reviews is 
considered as a review with a search conducted within last 
2 years). The EPC program provides EPCs with some guid- 
ance for assessing whether an existing review can answer 
key questions or sub-questions of a review. The EPC 
program recommends consideration of how recently the 
reviews primary literature search ended and how well the 
PICOTS-SD elements are reported and match with the 
current review, noting that partially relevant reviews may 
be useful for background or checking references. As de- 
scribed in Table 3, specific steps recommended in EPC 
guidance include conducting an initial screening for rele- 
vance while considering the timeliness of the review s lit- 
erature search. In the second stage of screening, the 
reviews PICOTS-SD elements should be compared to 
those in the new review protocol for relevance. If these 
elements are poorly reported, the new reviewers should 
not consider incorporating the existing review. If a re- 
view is outdated, but otherwise on point, an update of 
the search could be done. 

Evidence-based practice center discussions EPC mem- 
bers raised concerns about deciding which existing reviews, 
if any, to use in a new review, especially in situations when 
many reviews of varying quality or scope are available. It 
can often be difficult to find reviews that match all ele- 
ments of question. Still, existing reviews may be sufficiently 
similar to consider using, although criteria for all situations 
are not easy to define. Thus, determining which reviews 
are close enough' is inherently subjective and can be time 
consuming. 

Assessment Guidance exists for assessing the relevance of 
existing reviews for conducting overviews of reviews and 
for integrating existing reviews into a new review. Even so, 
EPC members expressed concerns about selecting which 
reviews to use when many are available. They reported 
rarely finding reviews that match directly all elements of 
questions. A clearer understanding of the factors (such as 
clinical area, review purpose, volume of literature, type of 



review) that have been correlated with successful (and un- 
successful) attempts to incorporate prior systematic reviews 
into new reviews would be helpful. 

Assessing review quality 

Available guidance Guidance is available on assessing the 
overall quality of systematic reviews (see Table 3). ECHTA 
and DACEHTA provide general guidance for assessing the 
quality of all literature, but do not specifically address asses- 
sing the quality of systematic reviews. In contrast, Institute 
for Quality and Efficiency in Health Care (IQWiG) and 
York Centre for Reviews and Dissemination (CRD) present 
guidance on assessing the quality of systematic reviews and 
recommend the use of a quality rating tool or checklist 
such as Oxman and Guyatt s quality index or the AMSTAR 
Instrument [4,5]. NICE suggests using existing reviews as 
evidence in developing a new guideline if they are up-to- 
date, have been assessed as high quality using the NICE 
methodology checklist for systematic reviews [15], and in- 
clude the accompanying evidence statements and evidence 
tables. EPC guidance recommends using AMSTAR as a 
starting point for quality assessment and supplementing as 
deemed appropriate for specific reviews. The Cochrane 
Collaboration is currently developing a new tool (ROBIS) 
to assess risk of bias in systematic reviews. In addition, the 
IOM report Finding What Works in Health Care: Stan- 
dards for Systematic Reviews may provide the basis for the 
development of instruments to assess quality of systematic 
reviews [11]. 

Evidence-based practice center discussions EPC 

members raised concerns about using AMSTAR to rate 
the quality of reviews, specifically, as to how accurate 
AMSTAR is in differentiating between poorer quality 
and better quality reviews. This is important since EPCs 
generally agreed that evidence from existing reviews 
should only be used if the review can be relied upon to 
substitute for the methods for the conduct of reviews es- 
poused by the EPC program. A suggestion was raised for 
a minimum standard of quality to be set for earlier re- 
views that are included as evidence in new reviews. 

Assessment Guidance is available for quality rating of 
reviews with available tools and checklists (for example, 
Oxman and Guyatt, AMSTAR, and National Institute 
for Health and Care Excellence (NICE)) [5,15,16]. How- 
ever, EPC members noted that these quality assessment 
tools have limitations. Further, it is important to remem- 
ber that these instruments do not assess the quality of 
primary evidence, which must also be taken into 
consideration. More explicit consideration of the quality 
rating approaches available for systematic reviews, and 
empiric evidence to support these, would help in the 
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consideration and selection of existing systematic re- 
views for use in new reviews. 

Determining appropriate use for relevant existing 
systematic reviews: scanning references 
Available guidance Cochrane, the EPC program, and 
CRD provide guidance on scanning the reference lists of 
existing systematic reviews (see Table 3). These organi- 
zations suggest scanning the reference lists of existing 
reviews as a supplementary method to find relevant 
studies for a new systematic review. Prior EPC program 
guidance suggested cross-checking primary studies from 
one or more existing systematic reviews to confirm ad- 
equacy of primary searches or to help select the most 
comprehensive existing review for incorporation. 

Evidence-based Practice Center Discussions Overall, 
most EPC members had experience using existing re- 
views for scanning references for relevant articles. No 
major concerns over the use of existing reviews in this 
manner were presented. 

Assessment Some guidance is available for using exist- 
ing reviews as a potential source of relevant studies for a 
new review. As noted below, incorporating search results 
from a prior systematic review to substitute for some or 
all of the new reviews search efforts is different from 
simply scanning references as an additional source of in- 
formation to support other efforts. More commonly, the 
latter occurs (that is, reference lists from existing reviews 
are used to augment or check comprehensiveness of the 
search yield used in the current review). In general, EPC 
members felt comfortable using existing reviews as a 
way to identify possible additional articles for a review. 
Thus, this does not appear to be an area where future 
guidance is needed except for the general methodo- 
logical question of when this type of hand-searching has 
reached saturation (that is, when no further relevant 
studies are being identified and searching may stop). 

Determining use: search strategy and results of existing 
searches 

Available guidance Cochrane and the EPC program pro- 
vide guidance on using the search strategy from existing re- 
views (see Table 3. The EPC program recommends using 
part or all of the search strategy from an existing review, 
particularly in the case of updating. The EPC program also 
indicates when it is acceptable to incorporate results of 
searches from existing systematic reviews to locate all pri- 
mary studies - that is, when the search methods (for 
example, search strategy, inclusion/exclusion criteria, litera- 
ture screening methods) are consistent with EPC program 
methods for finding evidence. The EPC program recom- 
mends supplementing primary studies from the existing 



systematic review by performing bridge searches whenever 
the search date for the existing review ended more than 
1 year earlier than searches for the new review are being 
conducted [2]. Data from primary studies gathered through 
these combined search efforts are then abstracted, followed 
by a new analysis and synthesis of data. Cochrane recom- 
mends utilizing information about search strategies from 
existing reviews and guidelines when possible in a new re- 
view, but does not address using search results in entirety. 

Evidence-based practice center discussions EPC mem- 
bers reported sometimes using the search strategy from 
an existing review, particularly when search strategies 
are reviewed and approved as robust by their medical 
librarians. EPCs also mentioned that, once they have 
determined it is methodologically appropriate to incorpor- 
ate an existing reviews search results, they go beyond sim- 
ply including the primary studies from the existing search. 
Instead, they check the primary studies against their inclu- 
sion/exclusion criteria, and some go further by also checking 
all studies from the excluded studies table in the previous 
systematic review, particularly in cases where there are 
slight differences in the questions or eligibility criteria. 

Assessment Some guidance is available on using the search 
strategy and search results from existing reviews. The EPC 
program provides some guidance for when to trust a 
search strategy from a prior review. Given the developing 
practice of requesting peer review for their search strat- 
egies, medical librarian review prior to incorporating an 
existing search strategy could be warranted [2]. Existing 
guidance related to when and how to incorporate the re- 
sults of previous search strategies is relatively robust in 
the current EPC guidance; incorporation of existing search 
results is also the most common way EPCs report incorp- 
orating existing systematic reviews into their work. Fur- 
ther guidance should be based on empiric evidence to 
establish whether any additional value is added by 1) con- 
sidering included studies from more than one existing sys- 
tematic review and 2) considering excluded studies from 
the previous systematic review upon which the search re- 
sults are being based. 

Determining use: data abstraction 

Available guidance Cochrane, NICE, and the EPC pro- 
gram provide guidance on using the data abstraction 
from existing reviews (see Table 3). When using existing 
reviews in a new guideline, NICE recommends creating 
new evidence summaries but using and directly referen- 
cing the original evidence tables from the prior review. 
For an overview of reviews, Cochrane guidance recom- 
mends using the abstracted data presented in the exist- 
ing reviews and seeking additional information from the 
authors of systematic reviews or from primary study 
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authors only if necessary. Though there is ongoing work 
in this area such as the Systematic Review Data Reposi- 
tory (SRDR) project, which will likely affect how ab- 
stracted data are used in the future, current guidance 
from the EPC program suggests using data abstraction 
tables from an existing review only if the methods used 
are deemed to be high quality and consistent with the 
EPC program methods for abstracting evidence. 

Evidence-based practice center discussions Most EPC 
members had concerns over whether or not data abstracted 
from an existing review are trustworthy, comprehensive, or 
accurate. Additionally, it was often reported that people 
tend to have to go back and abstract at least some data 
themselves from the primary studies due to slightly differ- 
ent outcomes, missing data or improper abstraction. This 
is particularly true when access to full evidence tables is 
limited, as can be the case with some peer-reviewed arti- 
cles. For any data abstraction that is incorporated, spot- 
checking for quality assurance was recommended, though 
there was no consensus on the minimum level required. 

Assessment Some guidance is available for using the 
data abstraction from existing reviews. However, there 
are still issues to be resolved when considering using ab- 
stracted data from an existing review. Additionally, on- 
going work such as the SRDR project could obviate the 
need for separate guidance if pre-abstracted, high-quality 
data become available for many reviews. At the least, 
tools like SRDR will result in the need for updated guid- 
ance which incorporates such new data resources for re- 
viewers. Guidance should specifically address various 
possible data abstraction scenarios. One such example is 
guidance related to including all versus only some of the 
abstracted data from an existing review. Because of differ- 
ent review scopes, PICOTS-SD, quality criteria of included 
studies, etcetera, systematic reviewers may determine that 
they will only need to use a portion of the data included in 
an existing review. Conversely, systematic reviewers may 
need to add to existing abstracted data either by abstract- 
ing additional data from previously included studies or by 
adding entirely new studies. Finally, as noted in the EPC 
discussion themes, systematic reviewers are often con- 
cerned about the trustworthiness or accuracy of abstracted 
data. Therefore, guidance is needed to establish consistent, 
reliable methods for conducting data checking or confirm- 
ing the abstracted data to assure accuracy. 

Determining use: study-level risk of bias assessments 
Available guidance Guidance is available on assessing the 
risk of bias of individual studies, though little guidance spe- 
cifically addresses how to incorporate this information from 
existing reviews into a new systematic review. AS noted in 
Table 3, Cochrane recommends conducting quality 



assessments of evidence for all outcomes addressed in an 
overview of reviews and evaluating the quality assess- 
ment judgments from existing reviews to ensure 
consistency among included reviews. If an existing re- 
view does not include a quality assessment, Cochrane 
recommends performing a quality assessment of the 
primary studies. 

Evidence-based practice center discussions EPC mem- 
bers noted that quality assessment of a systematic review to 
determine appropriateness for inclusion in a new systematic 
review is not sufficient to represent the quality of individual 
studies included in the existing review. When a review is in- 
corporated as more than a source of primary studies, it 
raises a number of issues about how to proceed in terms of 
quality rating. EPC members expressed strong concerns 
about relying on the risk of bias assessments from prior re- 
views; in particular, risk of bias assessments are known to 
be poorly reproducible, and often lack transparency and ra- 
tionale for ratings. No guidance is available on how to inte- 
grate information from reviews that do not have risk of bias 
assessments, or used risk of bias assessment tools different 
from those used in the new review. Overwhelmingly, EPC 
members felt that the primary studies included in an earlier 
review needed to be quality rated in some way that is suffi- 
ciently consistent with that applied to newly identified pri- 
mary studies included in the new review. Even if existing 
reviews include risk of bias assessments, an overarching 
theme from the discussion with EPC members was that it 
may be difficult for reviewers to trust the reliability of 
other reviewers' risk of bias assessments. Some EPC mem- 
bers felt that you may be able to trust particular sources 
(for example, Cochrane or EPCs) using particular risk of 
bias assessment tools. 

Assessment Guidance is readily available for assessing 
the risk of bias of primary studies with currently available 
tools and checklists (for example, Cochrane, EPC pro- 
gram) [2,8,17,18]. However, the lack of guidance about 
when to accept or when to repeat such assessments when 
conducted by others as part of prior reviews was of par- 
ticular concern to EPCs. Other issues of concern for EPCs 
include how to handle risk of bias assessment from prior 
reviews that are based on different tools from those used 
in a new review. Lack of clear guidance in this area is a 
barrier since the possible need to assess risk of bias for all 
primary studies introduces resource demands that reduce 
the likelihood of efficiency gains from incorporating exist- 
ing systematic reviews. 

Determining use: synthesis, including strength of evidence 
assessment 

Available guidance Cochrane and the EPC program 
provide some guidance on using the synthesis from 
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existing systematic reviews (see Table 3). Cochrane pro- 
vides guidance on using prior synthesis in an update of a 
review and in an overview of reviews. In an update, 
Cochrane recommends rerunning the quantitative syn- 
thesis if data from new studies are found. In an overview 
of reviews, Cochrane recommends relying on the previ- 
ous analysis as much as possible and only reanalyzing 
data if necessary (for example, if different populations or 
subgroups are analyzed in different reviews). The EPC 
program recommends incorporating an existing review 
in its entirety, including evidence summaries, only if it is 
high quality, the key questions are very similar to the new 
reviews questions, and the methods used are consistent 
with EPC program methods. However, current EPC pro- 
gram guidance does not explicitly address methods for 
incorporating prior syntheses. The EPC program does 
address dealing with discordant results from existing re- 
views and recommends making an effort to determine 
reasons for the disagreements both as part of the new 
review planning and in discussion of its findings. At the 
review planning stage, uncertainty of which discrepant 
results to trust can provide a strong rationale for con- 
ducting a new review. No current guidance is available 
on performing strength of evidence assessments for sys- 
tematic reviews, particularly as one component of a lar- 
ger body of evidence answering a key question. 

Evidence-based practice center discussions Most EPC 
members had concerns over how and if you can trust a 
prior review and whether or not you can trust the data 
synthesis. The lack of guidance available for assessing 
strength of evidence when integrating an existing system- 
atic review into a new review poses particular challenges. 
Despite current guidance, EPC members had additional 
concerns about how to deal with discordant results from 
reviews. 

Assessment Limited guidance is available on using the 
synthesis from existing reviews and this is an area that 
EPC members struggle with when considering using 
existing reviews. EPCs particularly struggle when asked 
to assess overall strength of evidence for mixed bodies 
of evidence (that is, those representing both primary 
studies and prior systematic reviews). The difficulty is 
compounded by the need for different guidance, depend- 
ing on the circumstances surrounding the prior review 
(s) and the current review. We identified four scenarios 
that might require different guidance as to how the syn- 
thesis may be used from an existing review: 1) up-to-date 
and has a meta-analysis; 2) up-to-date and presents a nar- 
rative summary; 3) not up-to-date and has a meta-analysis; 
and 4) not up-to-date and presents a narrative summary. 
These scenarios obviously apply only when one or more 
existing reviews are being included in a new review based 



on meeting certain criteria (for example, scope of key 
questions, PICOTS, and quality). 

Methods and results reporting 

Available guidance IQWiG, NICE, Cochrane and the 
EPC program present guidance on reporting results 
when utilizing existing reviews (see Table 3). The Annals 
of Internal Medicine has also published methodological 
reporting requirements for authors [19]. For a benefit 
assessment based on systematic reviews, IQWiG recom- 
mends summarizing the results from existing reviews in 
tables and noting any differences in results between re- 
views. In a guideline document using existing reviews, 
NICE recommends referencing evidence tables from exist- 
ing guidelines with a direct link to the source. Cochrane 
provides guidance on how to present information in an 
update of a review, including how to highlight information 
from new studies. When integrating existing reviews, the 
EPC program recommends including a summary table of 
included systematic reviews, including an assessment of 
the overlap in primary studies between reviews. Addition- 
ally, the EPC program recommends justifying the use of 
existing reviews in a new review and commenting on ad- 
vantages and disadvantages of using existing reviews in 
the discussion session. In an update of a review, the EPC 
program recommends explicitly showing what has chan- 
ged from the previous report. 

Evidence-based practice center discussions EPC mem- 
bers had concerns about how to show evidence from re- 
views, including how to highlight the primary evidence 
when using a prior review. Challenges have surfaced 
when users of EPC reports are unable to identify specific 
details about primary studies because they have been 
included as a part of a systematic review instead of 
highlighted individually. Some EPC members have pre- 
sented varying levels of detail based on how they used 
existing reviews but had no guidance on the best ap- 
proaches. Other EPC members have presented existing 
systematic reviews in tables and pulled out elements for 
discussion, even when the existing reviews are not other- 
wise integrated into a report. 

Assessment While some guidance is available for how 
to report the methods to locate and decide how to use 
an existing review, issues still may arise when reporting 
results from existing reviews. Specifically, it can be diffi- 
cult to highlight primary studies or isolate results from 
individual studies when integrating results from existing 
reviews. It is also important to not double count studies, 
such as through inclusion of primary studies also in- 
cluded in existing systematic reviews or by considering 
as separate evidence systematic reviews with overlapping 
primary studies. Guidance is needed on the level of 
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detail needed on all aspects, such as how new data were 
incorporated into evidence tables. Future examination of 
various approaches recommended by different groups 
and development of a set of templates with worked ex- 
amples could be very useful 

Conclusions 

It is considered good practice in research to explicitly 
consider what is already known. Specifically, there is also 
increased pressure to use existing systematic reviews in 
new reviews to leverage work already completed and 
thus, theoretically, reduce the amount of resources 
needed. Starting a new systematic review is resource- 
intensive, but it is unclear if the benefits of incorporating 
existing systematic reviews into new reviews are in some 
ways aspirational. This is perhaps in part because of the 
lack of methods for doing so. 

Our workgroup was formed as part of the methods 
work of the AHRQ EPC program and we focused on 
systematic reviews of health questions. Challenges in 
using existing systematic reviews in new reviews, and 
the options to address these challenges, may differ for 
different kinds of systematic reviews. We would suggest 
that examining whether our recommendations apply in 
these other contexts is an area for future work. How- 
ever, we feel that the concepts raised and discussed here 
are broadly applicable to systematic reviews in general. 



There is consensus among health-focused systematic 
review organizations and the EPCs about some as- 
pects of incorporating existing systematic reviews into 
new reviews. Best practice requires a discussion of 
existing systematic reviews in the introduction and 
discussion sections of any new review. In all reviews, 
it is important to discuss how the current systematic 
review conclusions concur or differ from previous re- 
views. There remain, however, challenges for review 
authors attempting to incorporate existing systematic 
reviews into new systematic reviews. The methodo- 
logical steps of using existing systematic reviews, sta- 
tus of existing guidance, and our recommendations 
are summarized in Table 4. 

Remaining areas of uncertainty may be addressed 
through a variety of methods. For instance, how best 
to construct evidence tables that combine primary 
studies and evidence from existing reviews may be 
addressed through discussions with end users of sys- 
tematic reviews. Methods research could assess the 
minimum quality control needed for using prior data 
abstractions or the impact on conclusions of a search 
of systematic reviews limited to specific sources. 
Finally, a symposium of systematic review organiza- 
tions may be needed to develop consensus on reporting 
standards for communicating how prior reviews are incor- 
porated. Future EPC program workgroups are taking up 
this challenge. 



Table 4 Guidance recommendations for methodological areas in using existing systematic reviews 



Methodological area 


Guidance status 


Recommendations/Further work needed 


Locating existing reviews 


Current guidance is time intensive 


Narrow search to locate only highly relevant, well-done, very recent, existing 
reviews; consider narrowing further to specific sources like EPC or Cochrane. 
Empiric study of comprehensive versus limited consideration of specific sources 
may be warranted. Consider how much documentation of search strategies and 
yields is required for transparency. 


Assessing relevance 


Guidance exists 


Follow current guidance 


Assessing review quality 


Current tools, such as AMSTAR, have 
limitations and none consider primary 
literature included in the reviews 


Empiric evidence of quality rating approaches is needed. Consider which currently 
available (or soon to be available) tools best fit the EPC program's needs. 


Determining use 






Scanning references 


Guidance exists 


Follow current guidance 


Search strategy/results 
of existing searches 


Guidance exists 


Empiric evidence for considering searches from >1 review and considering 
excluded studies is needed 


Data abstraction 


Current guidance is limited 


Guidance needed for specific scenarios and for confirming accuracy of 
abstracted data 


Study-level risk of bias 
assessments 


Guidance available for primary studies 


Guidance is needed for when to accept or repeat assessments from existing 
reviews 


Synthesis 


Current guidance is limited 


Guidance needed for specific scenarios and for assessing strength of evidence 
when integrating existing review 


Methods and results 
reporting 


Guidance exists 


Guidance needed on level of detail necessary for all aspects, and options with 
worked examples needed for evidence tables 



AMSTAR, A Measurement Tool to Assess Systematic Reviews; EPC, Evidence-based Practice Center. 
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